JavaSketchIt: Issues in Sketching the Look of User Interfaces

We present a visual approach to layout static components of user interfaces as hand-drawn compositions of simple geometric shapes, based on sketch recognition. We have defined a visual grammar using drawing data from target users, where we tried to figure out how people sketch interfaces and what combinations of shapes are more commonly used to define widgets. From these we built our grammar and implemented a prototype, JavaSketchIt, that allows creating user interfaces through hand-drawn geometric shapes, identified by a gesture recognizer. This prototype generates a Java interface, whose layout can be beautified using an a posteriori set of grammar rules (e.g. to align and group objects). To validate our approach, we conducted usability studies to compare our approach with a commercial system (JBuilder). Besides a measurable speed advantage in drawing interfaces, users found our system more comfortable, natural and intuitive to use, than the competing product, as demonstrated by post–experiment inquiries.